Methods and systems for signature verification

ABSTRACT

A method and system of signature verification that includes receiving a signature-in-question, extracting at least one of an image of the signature-in-question and a trajectory of the signature-in-question from the received signature-in-question, accessing a reference signature, extracting at least one of a reference image and a reference trajectory from the accessed reference signature, comparing the at least one of the image of the signature-in-question and trajectory of the signature-in-question to the at least one of the reference image and reference trajectory, and determining whether signature-in-question is authentic based on the comparison.

PRIORITY

This application claims the benefit of U.S. Utility application Ser. No.17/724,322, filed on Apr. 19, 2022, and titled “Methods and Systems forSignature Verification, which claims benefit of U.S. ProvisionalApplication No. 63/316,922, filed on Mar. 4, 2022, and titled “Methodsand Systems for Signature Verification,” the entire contents of whichare herein incorporated by reference.

BACKGROUND

Signature verification is a technique used by various entities such as,e.g., banks, to validate the identity of an individual. Signatureverification may be used to compare signatures in bank offices withsignatures captured in bank branches. An image of a signature, or anactual signature of, e.g., a bank customer, may be visually inspected orinstead fed into signature verification software to be compared to thesignature image on file in the bank office. Signature verificationsoftware is a type of software that compares signatures and checkswhether one of the signatures is authentic. This saves time and energy,helps decrease human error during the signature verification process,and lowers chances of signature fraud.

SUMMARY

In one aspect, the technology relates to a method of signatureverification that includes receiving a signature-in-question, extractingat least one of an image and a trajectory of the signature-in-question,accessing a reference signature file of a reference signature, thereference signature file including at least one of a reference image anda reference trajectory, extracting at least one of the reference imageand the reference trajectory from the accessed reference signature file,comparing the at least one of the image and trajectory of thesignature-in-question to the at least one of the reference image andreference trajectory, and determining whether the signature-in-questionis authentic based on the comparison.

In an example of the above aspect, the trajectory of thesignature-in-question and the reference trajectory respectively comprisetrajectory features including at least one of a sequence of coordinatesrepresenting locations of a writing member, a time stamp correspondingto each location, a pressure applied by the writing member, and a tiltof the writing member.

In yet another example of the above aspect, the writing member is one ofa stylus, a pen, and a finger of the person, and the tilt of the writingmember includes an altitude and an azimuth of the writing member. In anexample, at least one of a signing velocity, a signing acceleration, asigning pressure, and a signature time are derived from the trajectoryof the signature-in-question and the reference trajectory, respectively.

In further examples of the above aspect, receiving thesignature-in-question includes at least one of capturing thesignature-in-question on a signing device, receiving a photograph of thesignature-in-question, and receiving a scan of thesignature-in-question; the signing device includes one of a signaturepad and a signature screen; and receiving the signature-in-questionincludes receiving the signature-in-question at a kiosk, the kioskincluding at least one of a video camera, a photo camera, a scanner, anda signature pad.

In other examples of the above aspect, accessing the reference signaturefile includes accessing at least one of a scan of the referencesignature and a photograph of the reference signature; accessing thereference signature file may also include accessing the referencesignature file from a data repository; accessing the reference signaturefile from the data repository may include accessing the referencesignature file over a network; and extracting the reference trajectoryincludes extracting the reference trajectory from the memory.

In yet another example of the above aspect, the image of thesignature-in-question is received with the signature-in-question, theextracted reference trajectory is converted to the reference image, andthe image of the signature-in-question and the reference image arecompared to determine whether the signature-in-question is authentic.

In a further example of the above aspect, the trajectory of thesignature-in-question is received with the signature-in-question, thetrajectory of the signature-in-question is converted to the image of thesignature-in-question, and the image of the signature-in-question andthe reference image are compared to determine whether thesignature-in-question is authentic.

In another example of the above aspect, the trajectory of thesignature-in-question is received with the signature-in-question, andthe trajectory of the signature-in-question and the reference trajectoryare compared to determine whether the signature-in-question isauthentic.

In a further example of the above aspect, comparing the trajectory ofthe signature-in-question and the reference trajectory results in atleast one score; determining that the signature-in-question is a forgerywhen the at least one score is below a first threshold; and determiningthat a signing person is impaired when the at least one score is below asecond threshold.

In another example of the above aspect, the image of thesignature-in-question is received with the signature-in-question, theimage of the signature-in-question is converted to the trajectory of thesignature-in-question, and the trajectory of the signature-in-questionand the reference trajectory are compared to determine whether thesignature-in-question is authentic.

In a further example of the above aspect, the image of thesignature-in-question is received with the signature-in-question, andthe image of the signature-in-question and the reference trajectory arecompared to determine whether the signature-in-question is authentic.For example, comparing the image of the signature-in-question and thereference trajectory includes inputting the image of thesignature-in-question in a first neural network and generating a firstoutput; converting the reference trajectory to a reference image andinputting the reference image in a second neural network to generate asecond output; inputting the reference trajectory in a third neuralnetwork and generating a third output; concatenating the first input,the second input and the third input; inputting the concatenated first,second and third outputs into a classification neural network; andgenerating at least one score. As another example, at least one of thefirst output, the second output and the third output is a vector.

In other examples of the above aspect, comparing the image of thesignature-in-question and the reference trajectory includes inputtingthe image of the signature-in-question in a first neural network andgenerating a first output; inputting the reference trajectory in asecond neural network and generating a second output; and inputting thefirst output and the second output in a third neural network andgenerating at least one score.

In another example of the above aspect, the trajectory of thesignature-in-question is received with the signature-in-question, andthe trajectory of the signature-in-question and the reference image arecompared to determine whether the signature-in-question is authentic.For example, comparing the trajectory of the signature-in-question andthe reference image includes inputting the trajectory of thesignature-in-question in a first neural network and generating a firstoutput, inputting the reference image in a second neural network andgenerating a second output, inputting the trajectory of thesignature-in-question in a third neural network and generating a thirdoutput, concatenating the first output, the second output and the thirdoutput, inputting the concatenated first, second and third outputs intoa classification neural network, and generating at least one score.

In another aspect, the technology relates to a signature verificationsystem that includes a data receiver, an updatable data repositoryfunctionally coupled to the data receiver, a display device functionallycoupled to the data receiver, a processor operatively coupled to thedata receiver, to the updatable data repository and to the displaydevice, and a memory coupled to the processor, the memory storinginstructions. The instructions, when executed by the processor, performa set of operations including receiving, at the data receiver, asignature-in-question, extracting, via the processor, at least one of animage of the signature-in-question and a trajectory of thesignature-in-question from the received signature-in-question, accessinga reference signature file of a reference signature, the referencesignature file including at least one of a reference image and areference trajectory, extracting, via the processor, at least one of areference image and a reference trajectory from the accessed referencesignature, comparing, via the processor, the at least one of the imageof the signature-in-question and trajectory of the signature-in-questionto the at least one of the reference image and reference trajectory; anddetermining whether the signature-in-question is authentic based on thecomparison.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a system of signature verification, inaccordance with various aspects.

FIG. 2 is a flowchart describing a method of signature verification, inaccordance with various aspects.

FIGS. 3A-3B are flowcharts describing comparing an image of thesignature-in-question with the reference trajectory, in accordance withvarious aspects.

FIGS. 4A and 4B are flowcharts describing comparing the trajectory ofthe signature-in-question with the reference trajectory, in accordancewith various aspects.

FIGS. 5A-5D are flowcharts describing comparing the trajectory of thesignature-in-question with the reference trajectory, in accordance withvarious aspects.

FIG. 6 is an illustration of the effect of the tilt of a writing memberon a signature, in accordance with various aspects.

FIG. 7 depicts a block diagram of a computing device.

DETAILED DESCRIPTION

Determining whether a signature is authentic poses many challenges,particularly if the determination is to be performed in real time. Onechallenge is that authentic signatures made by the same person may havesome degree of variability. Another challenge is that the degree ofvariability between authentic signatures may vary from person to person.Yet another challenge is that forgers imitate signatures to attempt toobtain money, goods and/or services, and in the case of good forgeries,it may be difficult with the naked eye to determine whether a signatureis authentic or a forgery. Thus, the ability to improve the accuracy ofa verification process in real time may be advantageous. Accordingly,there is a need to reliably and accurately verify a signature todetermine whether the signature provided is authentic or whether it is aforgery. Another advantage may be to reliably and accurately verify thesignature in real time.

Accordingly, there is a technical problem that arises out of the factthat signature verification, e.g., in real time, is time consuming andinefficient. A solution to this technical problem may include receivinga signature-in-question from a person, extracting at least one of animage of the signature-in-question and a trajectory of thesignature-in-question from the received signature-in-question, accessinga reference signature, extracting at least one of a reference image anda reference trajectory from the accessed reference signature, comparingthe at least one of the image of the signature-in-question andtrajectory of the signature-in-question to the at least one of thereference image and reference trajectory, and determining whether thesignature-in-question is authentic based on the comparison.

Another solution to the problem is to rely on neural networks, which arecollections of weighted functions where the output of one weightedfunction or layer is the input in another weighted function or layer. Inthe case of a Siamese neural network, the neural subnetworks of theSiamese neural network have shared weights. The neural networks may betrained on known data sets of signatures, and trained neural networkscan then be used on unknown data sets of signatures. For example, theneural networks may be trained on a number of known signatures, as wellas on a number of known forgeries, so that when an unknown signature ispresented, it may be possible to determine with a degree of confidencewhether the unknown signature is in fact authentic or a forgery.

FIG. 1 is a diagram illustrating a method and system of signatureverification, in accordance with various aspects. In FIG. 1 , asignature 110 may be entered at a signing device 120, for example by aperson (not shown). In various aspects, the signature 110 may be enteredvia, e.g., a pen, a stylus, other writing member, a finger of theperson, or other method of entering a handwritten signature. In anotherexample, the signature 110 may be entered as an image such as, e.g., animage of an already existing signature on a photo ID document.Alternatively, several ID documents may be used to provide a signature.The signature 110 may also be entered in the form of a photograph or ascan. For example, an ID document may be provided in the form of a scan,and from the scan of the ID document, an image of the signature 110 maybe extracted. In an example, the signature 110 may be entered on asignature receiving device 120 such as a signature pad or signaturescreen or display such as, e.g., a handheld device, smartphone ortablet. The signature receiving device 120 may also be a kiosk includinga signature receiving surface, a video camera, a photo camera and/or ascanning device.

In various examples, the signature receiving device 120 may be connectedto a requesting entity 140 via a connection such as, e.g., a network130. For example, the network 130 may be the internet, a local areanetwork, a wide area network, a land line, a wireless network, or thelike. In various aspects, the requesting entity 140 may be, e.g., a bankor credit card company, a title company, or other organizationrequesting verification of the authenticity of a signature provided atthe signature receiving device 120. In operation, the requesting entity140 requests signature 110 for verification, and obtains the signature110, entered on the signature receiving device 120, over the network130. The requesting entity 140 is connected to a data repository 150 viaa connection, e.g., over a network 135. For example, the network 135 maybe the internet, a local area network, a wide area network, a land line,a wireless network, or the like. When the requesting entity 140 receivesthe signature 110, the requesting entity 140 may access data repository150. Upon accessing the data repository 150, the requesting entity 140may access a reference signature 160 from the data repository 150. Invarious examples, the data repository 150 may be, e.g., a governmentinstitution or the Department of Motor Vehicles (DMV), and therequesting entity 140 may obtain, from the data repository 150, areference signature 160 from the person alleged to have signed thesignature 110. Accordingly, both the signature 110 and the referencesignature 160 may be compared to determine whether the signature 110 issufficiently similar to the reference signature 160 to conclude that thesignature 110 is authentic, e.g., the signature 110 has been signed bythe same person who signed the reference signature 160. Determiningwhether the signatures 110 and 160 are from the same person is discussedin greater detail below.

FIG. 2 is a flowchart describing a method of signature verification, inaccordance with various aspects. For the sole purpose of convenience,method 200 is described through use of at least the example system 700described below. However, it is appreciated that the method 200 may beperformed by any suitable system.

Operation 210 includes receiving a signature-in-question, e.g., from aperson. For example, the person may be applying for a loan, foremployment, or for any other service, and may be submitting theirsignature for verification. In various aspects, receiving thesignature-in-question may be performed via, e.g., capturing thesignature-in-question on a signing device by the person physicallywriting their signature using a writing implement such as a pen, apencil, a stylus, and the like, receiving a photograph of thesignature-in-question, receiving a scan of the signature-in-question,receiving a drawing in place of the signature, or receiving a word orpassword in place of the signature. The signing device may be asignature pad or a signature screen such as, e.g., the screen of atablet, computer or smartphone. In other examples, receiving thesignature-in-question can be performed at a kiosk, where the kiosk mayhave a video camera, a photo camera, a scanner, a signature pad, or asignature screen such as the signature screen discussed above. Forexample, the signature may be signature 110, and the signature pad maybe similar to the receiving device 120, these elements being discussedabove with respect to FIG. 1 . In various aspects, thesignature-in-question may be received a plurality of times by the sameperson.

Operation 220 includes extracting an image of the signature-in-questionand/or a trajectory of the signature-in-question from the receivedsignature-in-question. For example, the trajectory of thesignature-in-question may have trajectory features which may include asequence of coordinates representing locations of a writing member on,e.g., the signature pad or screen, a time stamp corresponding to eachlocation of the writing member, a pressure applied by the person viawriting member, and/or a tilt of the writing member. In various aspects,the writing member may be a stylus, a pen, a pencil, and/or the fingerof the person providing the signature. In various aspects, for example,several additional features may be computed form the trajectory of thesignature-in-question, these features including, e.g., a signingvelocity, a signing acceleration, a signing pressure, a signature timeof the signature-in-question, and/or the tilt of the writing memberwhich includes an altitude and/or an azimuth of the writing member, asfurther discussed below in FIG. 6 .

Operation 230 includes accessing a reference signature file of areference signature, the reference signature file including a referenceimage and/or a reference trajectory. In various aspects, the referencesignature may have been previously provided by the person and kept as areference for authenticating later-provided signatures. In otheraspects, more than one reference signature may be stored. For example,any previously provided and approved signature may be stored. In anexample, the previously provided signature may also be a drawing inplace of the signature, or a word or password in place of the signature.In an aspect, accessing the reference signature file may be performed byaccessing a scan of the reference signature or a photograph of thereference signature that is stored in a data repository as the signaturefile. The reference signature may be accessed from a data repositorysuch as, e.g., a government agency such as the DMV, and the like, anemployer database, or another type of database or data repository. Forexample, the data repository may be similar to data repository 150discussed above with respect to FIG. 1 . The reference signature mayalso be accessed from the data repository over a network such as, aninternet connection, a wireless connection, a WAN or a LAN. For example,the network may be any one of networks 130 and 135 discussed above withrespect to FIG. 1 , and the government agency such as DMV and the like,employer database or other type of database may be similar to therequesting entity 140 discussed above with respect to FIG. 1 . Thereference signature may also be accessed on the same device as thedevice receiving the signature-in-question, or may be accessed in amemory that is directly connected to the device receiving thesignature-in-question. In other examples, when an image of an IDdocument is received as a reference signature, the image of thesignature itself may be extracted from the image of the ID document, andit is the extracted image of the signature that is accessed duringoperation 230.

Operation 240 includes extracting a reference image and/or a referencetrajectory from the accessed reference signature file. In variousaspects, the reference signature file is stored in a memory, or thereference signature may be encoded in the memory, for example asmetadata, and extracting the reference image/trajectory includesobtaining the reference image/trajectory from the memory or themetadata. As another example, the reference signature file may be an IDdocument, ID card or other card such as, e.g., a credit card, the IDcard or document may include a chip or magnetic strip on which thereference image and/or the reference trajectory may be encoded orstored.

Operation 250 includes comparing the image and/or the trajectory of thesignature-in-question to the reference image and/or the referencetrajectory. In an example of operation 250, the image of thesignature-in-question is received with the signature-in-question, uponaccessing the reference signature, the reference trajectory is extractedfrom the accessed reference signature file, the extracted referencetrajectory is converted to a reference image, and the image of thesignature-in-question and the converted reference image are compared todetermine, during operation 260, whether the signature-in-question isauthentic. Because the comparison in this aspect is between two images,the first and reference image may be compared using image comparisontechniques. This step is discussed in greater detail in the flowchartsof FIGS. 3A-3B. In various aspects, the signature pad includes a memoryand a processor, and the comparison during operation 250 may beperformed at the processor of the signature pad. Accordingly, privacymay be preserved for a person providing the signature.

FIGS. 3A-3B are flowcharts describing a method 300 of comparing an imageof the signature-in-question with the reference trajectory, inaccordance with various aspects. In FIG. 3A, in operation 310, asignature-in-question is received. This operation is similar tooperation 210 discussed above with respect to FIG. 2 and is notdiscussed further. In various aspects, the signature-in-question may bereceived a plurality of times by the same person. During operation 310,the signature-in-question is received as an image. During operation 330,the reference signature file is accessed. This operation is similar tooperation 230 discussed above with respect to FIG. 2 and is notdiscussed further. Upon accessing the reference signature file, thereference trajectory is extracted from the accessed reference signaturefile during operation 340. The extracted reference trajectory may thenbe converted, during operation 350, to a reference image, and duringoperation 360, the received image of the signature-in-question iscompared to the reference image. When the image of thesignature-in-question and the reference image are compared to eachother, then during operation 370, a determination is made whether thesignature-in-question is authentic. This operation is similar tooperation 260 discussed above with respect to FIG. 2 .

In another example of operation 250 in FIG. 2 , the trajectory of thesignature-in-question is received with the signature-in-question, thetrajectory of the signature-in-question is converted to the image of thesignature-in-question. Upon accessing the reference signature file, thereference image is extracted from the accessed reference signature, andthe image of the signature-in-question and the reference image arecompared to determine whether the signature-in-question is authentic atoperation 260. Because the comparison in this aspect is between twoimages, the image of the signature-in-question and the reference imagemay be compared using image comparison techniques. This step is alsodiscussed in greater detail in the flowchart of FIG. 3 .

In FIG. 3B, method 300 includes operation 310 where asignature-in-question is received as a trajectory. During operation 335,the trajectory of the signature-in-question is converted to an image ofthe signature-in-question. During operation 345, the reference signaturefile is accessed. This operation is similar to operation 330 discussedabove. Upon accessing the reference signature file, the reference imageis extracted from the accessed reference signature file during operation355. During operation 360, the image of the signature-in-question iscompared to the reference image. When the image of thesignature-in-question and the reference image are compared to eachother, then during operation 370, a determination is made whether thesignature-in-question is authentic. This operation is similar tooperation 260 discussed above with respect to FIG. 2 .

In yet another example of operation 250 from the flowchart of FIG. 2 ,the trajectory of the signature-in-question is received with thesignature-in-question, upon accessing the reference signature file, thereference trajectory is extracted from the accessed reference signaturefile, and the trajectory of the signature-in-question and the referencetrajectory are compared to determine whether the signature-in-questionis authentic at operation 260. In an example, comparing the trajectoryof the signature-in-question and the reference trajectory results in ascore. For example, the score may be indicative of various outcomes suchas, e.g., whether the signature-in-question is a forgery, whether thesignature-in-question is authentic, or whether the person was impairedat the time of producing the signature. The term “impaired” here meansthat the person may be cognitively impaired, under some type ofinfluence, suffering from an ailment or disease, or may be aging to thepoint of producing an altered signature. Accordingly, the score mayallow the determination of whether the signature-in-question is aforgery when the score is below a first threshold, whether the person isimpaired when the at score is below a second threshold, or whether thesignature-in-question is authentic and the person is not impaired whenthe score is above both the first and second thresholds, at operation260. This operation is discussed in greater detail in the flowcharts ofFIGS. 4A and 4B below.

In a further example of operation 250, the image of thesignature-in-question is received with the signature-in-question, theimage of the signature-in-question is converted to the trajectory of thesignature-in-question, the reference trajectory is extracted from theaccessed reference signature file, and the trajectory of thesignature-in-question and the reference trajectory are compared todetermine whether the signature-in-question is authentic at operation260. In various aspects, each of the trajectory of thesignature-in-question and the reference trajectory include a pluralityof trajectory features, and during operation 250, the trajectoryfeatures of each of the trajectory of the signature-in-question and thereference trajectory are compared to each other individually or as agroup. Alternatively, the trajectory of the signature-in-question andthe reference trajectory are compared by inputting both trajectories inseparate neural networks and analyzing the result. Details of convertingan image to a trajectory are taught in “Handwriting Trajectory Recoveryusing End-to-End Deep Encoder-Decoder Network” (Ayan Kumar Bhunia etal.; ICPR 2018) and “Automatic Trajectory Extraction and Validation ofScanned Handwritten Characters” (Ralph Niels and Louis Vuurpijl; October2006), which are incorporated herein by reference in their entirety.Details of the use of how to compare the trajectory of thesignature-in-question and the reference trajectory are further explainedin “State-of-the-Art in Handwritten Signature Verification System”(Rania A. Mohammed et al.; International Conference on ComputationalScience and Computational Intelligence; 2015), which is incorporatedherein by reference in its entirety. This operation is also discussed ingreater detail in the flowchart of FIGS. 4A and 4B.

FIGS. 4A and 4B are flowcharts describing a method 400 of comparing thetrajectory of the signature-in-question with the reference trajectory,in accordance with various aspects. In FIG. 4A, thesignature-in-question is received at operation 410. In various aspects,operation 420 determines whether the signature-in-question is receivedas an image or as a trajectory. At operation 435, thesignature-in-question is received as a trajectory of thesignature-in-question. At operation 445, the reference signature file isaccessed, the reference signature file including a reference trajectorystored or encoded therein, and at operation 455, the referencetrajectory is extracted from the accessed reference signature file.During operation 460, the trajectory of the signature-in-question andthe reference trajectory are compared to determine whether thesignature-in-question is authentic. In various aspects, each of thetrajectory of the signature-in-question and the reference trajectoryinclude a plurality of trajectory features, and during operation 460,the trajectory features of each of the trajectory of thesignature-in-question and the reference trajectory are compared to eachother individually or as a group. Alternatively, the trajectory of thesignature-in-question and the reference trajectory are compared byinputting both trajectories in separate neural networks and analyzingthe result. Details of the use of how to compare the trajectory of thesignature-in-question and the reference trajectory are further explainedin “State-of-the-Art in Handwritten Signature Verification System”(Rania A. Mohammed et al.; International Conference on ComputationalScience and Computational Intelligence; 2015), which is incorporatedherein by reference in its entirety. In an example, based on thecomparison between the trajectory of the signature-in-question and thereference trajectory during operation 460, a score is generated atoperation 470.

Alternatively, as also illustrated in the flow chart of FIG. 4A, whenthe signature-in-question is received at operation 410 and operation 420determines that the signature-in-question is received as an image, atoperation 430, the image of the signature-in-question is converted tothe trajectory of the signature-in-question. During operation 440, thereference signature file is accessed, and during operation 450, thereference trajectory is extracted from the accessed reference signaturefile. During operation 460, the trajectory of the signature-in-questionand the reference trajectory are compared, and based on the comparisonbetween the trajectory of the signature-in-question and the referencetrajectory during operation 460, a score is generated at operation 470.Once the score is generated at operation 470, the method 400 continuesto operations 480-499 to analyze the generated score.

FIG. 4B is a flowchart describing additional steps of the method 400, inaccordance with various aspects. In FIG. 4B, for example, the scoregenerated during operation 470 may be indicative of various outcomessuch as, e.g., whether the signature-in-question is a forgery, whetherthe signature-in-question is authentic but the person is impaired at thetime of producing the signature, or whether the signature-in-question isauthentic and the person is not impaired. For example, if the score isbelow a first threshold, as determined during operation 480, then it maybe concluded at operation 485 that the signature-in-question is aforgery. In an example, if the score is equal to or above the firstthreshold, as determined in operation 480, the score may be compared toa second threshold during operation 490. During operation 490, if thescore is lower than the second threshold, the second threshold beinghigher than the first threshold, then it may be determined at operation495 that the signature-in-question is authentic but the person isimpaired. In other aspects, if the score is equal to or higher than thesecond threshold, as determined during operation 490, then it may bedetermined during operation 499 that the signature-in-question isauthentic and that the person producing the signature-in-question is notimpaired. Accordingly, the score generated during operation 470 mayallow the determination of whether the signature-in-question isauthentic, whether the person who produced the signature is impaired, orwhether the signature-in-question is a forgery. In various aspects, ifthe score is determined to be inconclusive or too low, then operations410 through 499 may be repeated more than once until a conclusion as tothe authenticity of the signature-in-question may be assessed.

In another example of operation of the flow chart of FIG. 2 , operations210-240 are as discussed above, and comparing the image of thesignature-in-question and the reference trajectory during operation 250may be performed by inputting the image of the signature-in-question ina first neural network and generating a first output, converting thereference trajectory to a reference image and inputting the referenceimage in a second neural network and generating a second output. Thefirst and second output may be two vectors of the same dimensionality.The comparison also includes inputting one or more trajectory featuresof the reference trajectory in a third neural network and generating athird output, the third output being, e.g., a vector of a predetermineddimension. The comparison also includes concatenating the first output,the second output and the third output, and inputting the concatenatedoutputs into a classification neural network, for example a perceptron,to generate at least one score representing a degree of confidence thata given pair of signatures belong to the same person.

In various aspects, the neural networks may be trained jointly on datasets where the reference signature and the signature-in-question areproduced by the same person, and data sets where the reference signatureand the signature-in-question are produced by different persons. Forexample, the first and second neural networks may be subnetworks of aSiamese neural network. Details of the use of the neural networks tocompare image and image are further explained in “SigNet: ConvolutionalSiamese Network for Writer Independent Offline Signature Verification”(Sounak Dey et al.; Pattern Recognition letters, 30 Sep. 2017), which isincorporated herein by reference in its entirety. This operation isdiscussed in greater detail in the flowcharts of FIGS. 5A-5D below.

In another example of operation 250, comparing the image of thesignature-in-question and the reference trajectory may be performed byinputting the image of the signature-in-question in a first neuralnetwork and generating a first output in the form of a first vector,inputting the reference trajectory in a second neural network andgenerating a second output in the form of a second vector, and inputtingthe first output and the second output in a third neural network thatcompares the first vector and the second vector to generate at least onescore as a result. In various aspects, the three neural networks may bejointly trained on data set that consist of cases where the referencesignature and the signature-in-question belong to the same person, andof cases where the reference signature and the signature-in-questionbelong to different persons. For example, the first two neural networksmay be trained to perform signature comparison and generate embeddingvectors, and the third neural network may be trained to compareembedding vectors and generate a score indicative of a degree ofconfidence that the signatures are from the same person or fromdifferent persons. This operation is also discussed in greater detail inthe flowcharts of FIGS. 5A-5D below.

FIGS. 5A-5D are flowcharts describing comparing the image of thesignature-in-question with the reference trajectory, in accordance withvarious aspects. In FIG. 5A, the image of the signature-in-question isreceived with the signature-in-question during operation 510. Duringoperation 520, the reference signature file is accessed, and duringoperation 530, the reference trajectory is extracted from the accessedreference signature file. During operation 540, the image of thesignature-in-question is compared to the reference trajectory, asfurther described in FIG. 5B, in accordance with various aspects. Basedon the comparison performed during operation 540, a score is generatedduring operation 550.

Referring back to operation 540 in FIG. 5A, the image of thesignature-in-question and the reference trajectory are compared to eachother to determine whether the signature-in-question is authentic. InFIG. 5B, which is a more detailed description of operation 540,comparing the image of the signature-in-question and the referencetrajectory may be performed by inputting the image of thesignature-in-question in a first neural network during operation 560. Asa result, during operation 561, a first output is generated. Forexample, the first output may be in the form of a vector. Duringoperation 562, the reference trajectory is converted to a referenceimage, and during operation 563, the reference image is inputted in asecond neural network. As a result of the input of the reference imagein the second neural network, at operation 564, a second output isgenerated. For example, the second output is in the form of anothervector. The first and second outputs may be two vectors of the samedimensionality. During operation 565, one or more trajectory features ofthe reference trajectory are inputted in a third neural network, andduring operation 566, a third output is generated as a result, the thirdoutput being, e.g., a vector of a predetermined dimension. As furtherdescribed in FIG. 5C, once the first output, second output and thirdoutput are generated, during operation 570, the first output, the secondoutput and the third output are concatenated together. During operation572, the concatenated outputs are inputted into a classification neuralnetwork, or into a perceptron, and during operation 574, at least onescore is generated, the at least one score representing a degree ofconfidence that a given pair of signatures belong to the same person.

After the score is generating during operation 550, or during operation574, the score may be indicative of various outcomes such as, e.g.,whether the signature-in-question is a forgery, whether thesignature-in-question is authentic but the person is impaired at thetime of producing the signature, or whether the signature-in-question isauthentic and the person is not impaired. For example, as illustrated inFIG. 5D, during operation 580, a determination is made whether the scoreis below a first threshold. If the score is below the first threshold,then it may be concluded at operation 585 that the signature-in-questionis a forgery. In an example, if the score is equal to or above the firstthreshold, as determined in operation 580, the score may then becompared to a second threshold during operation 590. During operation590, if the score is lower than a second threshold greater than thefirst threshold, then it may be determined at operation 595 thatalthough the signature-in-question is authentic, but that the person whoproduced the signature-in-question is impaired. If the score is equal toor higher than the second threshold, as determined during operation 590,then it may be determined during operation 599 that thesignature-in-question is authentic and the person producing thesignature-in-question is not impaired. In various aspects, if the scoreis determined to be inconclusive or too low, then the operations 510-599may be repeated until a conclusive score may be determined. In variousaspects, although the above operations describe comparing the image ofthe signature-in-question with the reference trajectory, the sameoperations may apply to comparing the trajectory of thesignature-in-question with the reference image.

FIG. 6 is an illustration of the effect of the tilt of a writing memberon the signature, in accordance with various aspects. In FIG. 6 , anillustrative handwritten signature may be detected by a writing member610, e.g., with a digitizer that allows pen events to be continuouslydetected when the writing member 610 is lifted from the surface of thedigitizer. Accordingly, in the example shown in FIG. 6 , rotation,altitude, and azimuth are among the parameters that can be detected forthe writing member 610.

FIG. 7 depicts a block diagram of a computing device, according tovarious aspects. In the illustrated example, the computing device 700may include a bus 702 or other communication mechanism of similarfunction for communicating information, and at least one processingelement 704 (collectively referred to as processing element 704) coupledwith bus 702 for processing information. As will be appreciated by thoseskilled in the art, the processing element 704 may include a pluralityof processing elements or cores, which may be packaged as a singleprocessor or in a distributed arrangement. Furthermore, a plurality ofvirtual processing elements 704 may be included in the computing device700 to provide the control or management operations for the displays 120and 160 or the methods 300, 400 and 500 illustrated above.

The computing device 700 may also include one or more volatilememory(ies) 706, which can for example include random access memory(ies)(RAM) or other dynamic memory component(s), coupled to one or morebusses 702 for use by the at least one processing element 704. Computingdevice 700 may further include static, non-volatile memory(ies) 708,such as read only memory (ROM) or other static memory components,coupled to busses 702 for storing information and instructions for useby the at least one processing element 704. A storage component 710,such as a storage disk or storage memory, may be provided for storinginformation and instructions for use by the at least one processingelement 704. As will be appreciated, the computing device 700 mayinclude a distributed storage component 712, such as a networked disk orother storage resource available to the computing device 700.

The computing device 700 may be coupled to one or more displays 714 fordisplaying information to a user. Optional user input device(s) 716,such as a keyboard and/or touchscreen, may be coupled to Bus 702 forcommunicating information and command selections to the at least oneprocessing element 704. An optional cursor control or graphical inputdevice 718, such as a mouse, a trackball or cursor direction keys forcommunicating graphical user interface information and commandselections to the at least one processing element. The computing device700 may further include an input/output (I/O) component, such as aserial connection, digital connection, network connection, or otherinput/output component for allowing intercommunication with othercomputing components and the various components of the displays 120 and160 or the methods 300, 400 and 500 illustrated above.

In various embodiments, computing device 700 can be connected to one ormore other computer systems via a network to form a networked system.Such networks can for example include one or more private networks orpublic networks, such as the Internet. In the networked system, one ormore computer systems can store and serve the data to other computersystems. The one or more computer systems that store and serve the datacan be referred to as servers or the cloud in a cloud computingscenario. The one or more computer systems can include one or more webservers, for example. The other computer systems that send and receivedata to and from the servers or the cloud can be referred to as clientor cloud devices, for example. Various operations of the displays 120and 160 or the methods 300, 400 and 500 illustrated above may besupported by operation of the distributed computing systems.

The computing device 700 may be operative to control operation of thecomponents of the displays 120 and 160 or the methods 300, 400 and 500illustrated above through a communication device such as, e.g.,communication device 720, and to handle data provided from the datasources as discussed above with respect to the displays 120 and 160 orthe methods 300, 400 and 500. In some examples, analysis results areprovided by the computing device 700 in response to the at least oneprocessing element 704 executing instructions contained in memory 706 or708 and performing operations on the received data items. Execution ofinstructions contained in memory 706 and/or 708 by the at least oneprocessing element 704 can render the displays 120 and 160 or themethods 300, 400 and 500 operative to perform methods described herein.

The term “computer-readable medium” as used herein refers to any mediathat participates in providing instructions to the processing element704 for execution. Such a medium may take many forms, including but notlimited to, non-volatile media, volatile media, and transmission media.Non-volatile media includes, for example, optical or magnetic disks,such as disk storage 710. Volatile media includes dynamic memory, suchas memory 706. Transmission media includes coaxial cables, copper wire,and fiber optics, including the wires that include bus 702.

Common forms of computer-readable media or computer program productsinclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, digital videodisc (DVD), a Blu-ray Disc, any other optical medium, a thumb drive, amemory card, a RAM, PROM, and EPROM, a FLASH-EPROM, any other memorychip or cartridge, or any other tangible medium from which a computercan read.

Various forms of computer readable media may be involved in carrying oneor more sequences of one or more instructions to the processing element704 for execution. For example, the instructions may initially becarried on the magnetic disk of a remote computer. The remote computercan load the instructions into its dynamic memory and send theinstructions over a telephone line using a modem. A modem local tocomputing device 700 can receive the data on the telephone line and usean infra-red transmitter to convert the data to an infra-red signal. Aninfra-red detector coupled to bus 702 can receive the data carried inthe infra-red signal and place the data on bus 702. Bus 702 carries thedata to memory 706, from which the processing element 704 retrieves andexecutes the instructions. The instructions received by memory 706and/or memory 708 may optionally be stored on storage device 710 eitherbefore or after execution by the processing element 704.

In accordance with various embodiments, instructions operative to beexecuted by a processing element to perform a method are stored on acomputer-readable medium. The computer-readable medium can be a devicethat stores digital information. For example, a computer-readable mediumincludes a compact disc read-only memory (CD-ROM) as is known in the artfor storing software. The computer-readable medium is accessed by aprocessor suitable for executing instructions configured to be executed.

This disclosure described some examples of the present technology withreference to the accompanying drawings, in which only some of thepossible examples were shown. Other aspects can, however, be embodied inmany different forms and should not be construed as limited to theexamples set forth herein. Rather, these examples were provided so thatthis disclosure was thorough and complete and fully conveyed the scopeof the possible examples to those skilled in the art.

Although specific examples were described herein, the scope of thetechnology is not limited to those specific examples. One skilled in theart will recognize other examples or improvements that are within thescope of the present technology. Therefore, the specific structure,acts, or media are disclosed only as illustrative examples. Examplesaccording to the technology may also combine elements or components ofthose that are disclosed in general but not expressly exemplified incombination, unless otherwise stated herein. The scope of the technologyis defined by the following claims and any equivalents therein.

What is claimed is:
 1. A method of signature verification, the methodcomprising: receiving a signature-in-question; extracting at least oneof an image of the signature-in-question and a trajectory of thesignature-in-question from the received signature-in-question; accessinga reference signature from a reference signature storage, the referencesignature storage including at least one of a reference image and areference trajectory; extracting at least one of the reference image andthe reference trajectory from the accessed reference signature storage;and comparing the at least one of the image of the signature-in-questionand trajectory of the signature-in-question to the at least one of thereference image and reference trajectory.
 2. The method of claim 1,further comprising determining whether the signature-in-question isauthentic based on the comparison.
 3. The method of claim 1, wherein atleast one of a signing velocity, a signing acceleration, a signingpressure, and a signature time of the signature-in-question and thereference signature are derived from at least one of the trajectory ofthe signature-in-question and the reference trajectory.
 4. The method ofclaim 1, wherein the receiving of the signature-in-question comprises atleast one of capturing the signature-in-question on a signing device,receiving a photograph of the signature-in-question, receiving a scan ofthe signature-in-question, receiving a drawing as thesignature-in-question, and receiving a word or a password as thesignature-in-question.
 5. The method of claim 4, wherein the accessingthe reference signature storage comprises accessing the referencesignature storage from the signing device.
 6. The method of claim 1,wherein the accessing the reference signature storage of the referencesignature comprises accessing at least one of a scan of an identitydocument, a photograph of the identity document, a scan of the referencesignature, a photograph of the reference signature, a drawing as thereference signature, and a word or password as the reference signature.7. The method of claim 1, wherein the accessing the reference signaturestorage comprises accessing the reference signature storage from one ofa data repository, a memory or magnetic strip of an identificationdocument, and a memory or magnetic strip of a magnetic card.
 8. Themethod of claim 1, wherein: the image of the signature-in-question isreceived with the signature-in-question; the extracted referencetrajectory is converted to the reference image; and the image of thesignature-in-question and the reference image are compared to determinewhether the signature-in-question is authentic.
 9. The method of claim1, wherein: the trajectory of the signature-in-question is received withthe signature-in-question; and the trajectory of thesignature-in-question and the reference image are compared to determinewhether the signature-in-question is authentic.
 10. The method of claim9, wherein the trajectory of the signature-in-question and the referenceimage are compared without converting the trajectory of thesignature-in-question to an image thereof.
 11. The method of claim 9,wherein the reference image is extracted from an identity documentimage.
 12. A signature verification system comprising: a data receiver;an updatable data repository functionally coupled to the data receiver;a display device functionally coupled to the data receiver; a processoroperatively coupled to the data receiver, to the updatable datarepository and to the display device; and a memory coupled to theprocessor, the memory storing instructions that, when executed by theprocessor, perform a set of operations comprising: receiving, at thedata receiver, a signature-in-question; extracting, via the processor,at least one of an image of the signature-in-question and a trajectoryof the signature-in-question from the received signature-in-question;accessing a reference signature storage of a reference signature, thereference signature storage including at least one of a reference imageand a reference trajectory; extracting, via the processor, at least oneof the reference image and the reference trajectory from the accessedreference signature storage; and comparing, via the processor, the atleast one of the image of the signature-in-question and trajectory ofthe signature-in-question to the at least one of the reference image andreference trajectory.
 13. The system of claim 12, wherein the set ofoperations further comprise determining whether thesignature-in-question is authentic based on the comparison.
 14. Thesystem of claim 12, wherein the trajectory of the signature-in-questionand the reference trajectory respectively comprise trajectory featuresincluding at least one of a sequence of coordinates representinglocations of a writing member, a time stamp corresponding to eachlocation, a pressure applied by the writing member, and a tilt of thewriting member.
 15. The system of claim 14, wherein the writing memberis one of a stylus, a pen, and a finger of a signing person.
 16. Thesystem of claim 12, the signature-in-question is at least one ofcaptured on a signing device, received as a photograph thereof, andreceived as a scan thereof.
 17. The system of claim 12, wherein thesignature-in-question is received at a kiosk, the kiosk including atleast one of a video camera, a photo camera, a scanner, and a signaturepad.
 18. The system of claim 12, wherein the reference signature isaccessed via at least one of a scan of an identity document, aphotograph of the identity document, a scan of the reference signature,a photograph of the reference signature, a drawing as the referencesignature, and a word or password as the reference signature.
 19. Thesystem of claim 12, wherein: the trajectory of the signature-in-questionis received with the signature-in-question; and the set of operationscomprises comparing the trajectory of the signature-in-question and thereference image to determine whether the signature-in-question isauthentic.
 20. The system of claim 19, wherein the set of operationscomprises comparing the trajectory of the signature-in-question and thereference image without converting the trajectory of thesignature-in-question to an image thereof.
 21. The system of claim 19,wherein the set of operations comprise extracting the reference imagefrom an identity document image.